The current situation raises many questions for people whose work involves data collection. These need to be considered on a case by case basis and reviewed as the situation develops. Therefore, instead of providing definitive guidance, we have set out some issues analysts should consider when deciding how to proceed with current, planned or new data collection exercises. This will be kept up to date as the situation develops.
Face-to-face data collection
The advice at the end of March 2020 was clear and unambiguous - that all face-to-face data collection should be suspended immediately on the grounds of public safety. At the point current restrictions are relaxed, project managers should still consider carefully the ethical, practical and methodological implications of travel and face-to-face data collection. If remote (e.g. online, telephone) data collection is used as an alternative this may also impose additional pressures and risks.
Ethical considerations
It is essential to consider the impact of the current situation on your target population. Some businesses, local authorities and individuals will currently be experiencing very difficult circumstances, so it may not be appropriate to impose additional burdens on them. It may also seem very out of touch; consider the reputational risk to continuing in these circumstances.
- Is it appropriate and proportionate for you to be contacting respondents at this time?
- Are there any risks to the health and well-being of data providers if the data are collected now?
- Will interviewers require additional training, e.g. if working with individuals, businesses or local authorities in distress?
Practical considerations
The current situation may pose practical challenges in data collection, not least of which is whether the research is still relevant. Given that a research question remains pertinent:
- Do data providers have the resources to dedicate to the research?
- Are data providers able to access the systems they need to provide you with the data?
- Can you still access (a sufficient, representative sample of) respondents?
Methodological/quality considerations
If you continue with your data collection, consider if the results will be of sufficient quality to be usable?
- For example, might the current situation influence responses rates in a way which introduces potential bias?
- Might the situation influence responses such that results are not generalisable to ‘normal times’?
- If you change the mode of data collection (e.g. from face-to-face to online) what are the likely impacts on your data and implications for interpretation (e.g. we know mode effects can be large)?
Value
Data collection that monitors the current situation, contributes to an immediate understanding of the crisis or informs decision making throughout the crisis should be prioritised.
- Is this data required for COVID19 understanding, decision making, funding and payments or analysis?
- Is this data required to inform our understanding of changes in the fabric of the economy or society due to Covid-19 (particularly any immediate changes)?
- Are operational/policy decisions made directly from this data?
- How will stopping collection impact on the quality of those decisions?
- Is the data part of a longer-term time series
For some regular series, data collection may not currently be a top priority but, providing other caveats (ethical, practical etc) have been considered, continuing collection may be justifiable to maintain the time series.
Research Options - Reducing the Burden on Respondents
Taking into account these considerations, there may be various options for proceeding, from continuing as before to stopping altogether. Within this, there are various changes or alternatives to consider to reduce the burden on respondents. For example:
- Are there alternative data sources for this data collection? Is there a viable alternative with enough quality to meet user needs for now?
- Is there anything you can do to reduce burden that will keep critical data flowing? For example, can you reduce the scope or complexity of the data requested or use auxiliary information to model missing data?
- Is data needed from the full population (of Local Authorities, for example) or would a sampling approach be sufficient?
- Can the data be collected at a later stage?
- Can the frequency of collection be reduced? What are the consequences of reducing frequency?
- Can you give flexibility over when data providers supply the data? Some may find it straightforward and easier to supply quarter by quarter, whereas others may prefer to submit two quarters of data once business returns to normal
- Are there alternative options for asking your research questions, e.g. multi-purpose data collection vehicles such as commercial omnibuses that reduce respondent burden?
Acknowledgements
The Government Social Research profession is part of the civil service and supports the development, implementation, review and evaluation of government policy. This blog post is based on internal guidance produced by GSR Heads of Profession in April 2020. With thanks to the GSR leadership for sharing the guidance with SRA and approving this version.
Jenny Dibden, Head of GSR and Co-Director Cities and Local Growth Unit at Ministry of Housing, Communities and Local Government
Siobahn Campbell, Deputy Head of GSR and Head of the Central Research Team, Deputy Chief Scientific Adviser, Department for Transport
Ed Dunn, Deputy Head of GSR and Deputy Director Social Surveys & Transformation, Office for National Statistics